The proliferation of internet- and mobile-connected devices—the ‘Internet of Everything’—has increased data traffic volume, transmission speeds and usage on communications networks. The ubiquity of device types and connections (cellular, wireless, multi-SIM, machine-to-machine) and the expansion of usage types (voice, high-definition video, music, data) have also made it more complex to monitor and secure these networks and to conduct analysis on the traffic and content.
To accomplish this, the traffic must be instrumented (what data is moving across the network), analyzed (what is the content of the traffic), and understood (what are the implications of this) so a relevant decision can be made or action taken within the available window of opportunity. This is especially so in the case of time-critical revenue, customer, operational, or security impacting events. Examples of such events include fraud occurring on mobile carrier networks, cellular zones dropping calls above an acceptable threshold, malfunctioning mobile applications, or malicious content or agents compromising a network.
This network data is captured by a variety of network probes sitting ‘inline’ (intrusively) inside the network. Network events must first ‘complete’ (example: after a voice call is completed and goes through ‘call teardown’) before they are translated into offline database records (example: Call Detail Records, Event Detail Records). These records are extracted at regular time intervals and provided to applications in offline enterprise data centers for post-event processing and analysis.
These systems can suffer from latency delays of up to 15 minutes for event data to be extracted and delivered to databases. In many cases, multiple terabytes of data are written into databases, posing ‘Big Data’ analytical challenges when time-critical results are needed. The inline hardware represents significant capital expenditures. These types of systems also provide a limited ability to respond flexibly to live conditions, as the application layer is not integrated contextually within the data collection layer. Database records are not generated for some network events that may provide indications of fraud or other critical issues that must be detected.
A use case is mobile carrier fraud detection that utilizes call detail records that have been delivered to a data warehouse after the relevant network traffic or calls have been completed. Detection of fraud in this case occurs after the actual fraudulent even has occurred, and in many cases, the carrier has already incurred a financial loss. Any actions taken to remediate (example: block the caller) can only be applied to the next time a relevant event appears in the network.
The present invention, as disclosed and described herein, in one aspect thereof, comprises a system for monitoring a live-data flow through a network. The system includes at least one server communication with the network and at least one network interface associated with the at least one server for providing access to the live-data flow through the network. A processor within each of the at least one serve implements a first processing node for monitoring a mirrored live-data flow of the live-data flow passing through at least one selected point within the network in a non-intrusive manner that does not affect the live-data flow passing through the at least one selected point. The live-data flow comprises a plurality of simultaneous live-data flows that are in active transmission between endpoints in the network and prior to onward storage of the data in a database. The first processing node decodes data within the mirrored live-data flow according to each protocol associated with the data. The data has a plurality of protocols associated therewith, and the data is decoded in parallel according to each of the plurality of protocols. The first processing node detects at least one predetermined or deduced condition defined by at least one of a plurality of applications implemented on a second processing node. The first processing node also executes at least one predetermined or deduced response responsive to an indication of occurrence of the at least one predetermined or deduced condition within the decoded data. The first processing node also forwards data from the first processing node to a second processing node data from at least one of the plurality of simultaneous live-data flows based upon occurrence of the at least one predetermined or deducted condition defined by the at least one of the plurality of applications implements on the second processing node. The processor within the at least one server and the processor further implements the second processing node for accessing from the second processing node, external data from an external data source. The second processing node also processes at least a portion of the data forwarded from the first processing node using at least one of the plurality of applications implemented on the second processing node and the external data. The processing of the data by the at least of the plurality of applications and the external data causes execution of the at least one predetermined or deduced response to determine a manner for controlling an operation of the network at a same time the live-data flow is in active transmission between the endpoints in the network. The operation of the network is then controlled in response to the executed at least one predetermined or deduced response while events associated with the live-data flow are occurring within the network.